There has been substantial growth in the purchase of information technology as a service from internal and external service providers and this trend appears to be increasing rapidly. This growth is enabled by the trend towards cloud computing, in which, services run on shared virtualized resource pools that are accessible via Intranets or the Internet. As this cloud computing paradigm matures, there is also an increasing trend for businesses to exploit the paradigm to support business critical services such as sales and delivery, and supply chain management. Those services will have performance requirements and are likely to place significant loads on cloud infrastructures.
With the increasing loads currently being placed on cloud computing infrastructures, it is becoming increasingly important to evaluate the cloud computing infrastructures to explore the capabilities of the systems through testing. For instance, testing of the cloud computing infrastructures to evaluate the performance impacts of running various services in the cloud environments, as well as getting an overview of the individual performance of parts of the system, would provide beneficial understanding of the systems in the cloud computing infrastructures.
In addition, it is becoming increasingly important to create test systems configured to accurately model the workloads imposed upon the systems contained in the cloud computing infrastructures. One modeling method utilizes benchmarks to impose a synthetic workload on the cloud computing infrastructures being tested. A typical business process, for instance, ordering, may invoke a number of discreet business objects in order to complete the business process. In addition, a given business object may be characterized by a particular sequence of interdependent requests which are exchanged between entities in the enterprise application system. In other words, the sequence of interdependent requests should be performed correctly in order to correctly implement the business process. Thus, a benchmark for modeling the enterprise application system should accurately reflect the correct sequence and volume of interdependent requests. Otherwise, an incorrect sequence of interdependent requests may cause an error condition that does not accurately model the demands placed upon the enterprise application system.
However, conventional stress testing of cloud computing infrastructures is generally based upon a small number of pre-existing benchmarks which typically utilize a small subset of business objects. As a result, it is difficult to generate a synthetic workload which accurately models the actual load patterns expected on the cloud computing infrastructures. Alternatively, creating a customized benchmark which is representative of a given enterprise is typically too time consuming and expensive for many users.
It would thus be beneficial to have methods for verifying that services can be provided at required throughput levels while using an acceptable quantity of resources.